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30  NOVEMBER  -  2  DECEMBER  1982 
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Sponsored  by: 


Ho* tod  by: 


Air  Foret  Systems  Command 


Aeronautical  Systems  Division 


FOREWORD 


THE  UNITED  STATES  AIR  FORCE  HAS  COMMITTED  ITSELF  TO  "STANDARDIZATION.” 
THE  THEME  OF  THIS  YEAR’S  CONFERENCE  IS  "RATIONAL  STANDARDIZATION,"  AND  WE 
HAVE  EXPANDED  THE  SCOPE  TO  INCLUDE  US  ARMY,  US  NAVY  AND  NATO  PERSPECTIVES 
ON  ONGOING  DOD  INITIATIVES  IN  THIS  IMPORTANT  AREA. 


WHY  DOES  THE  AIR  FORCE  SYSTEMS  COMMAND  SPONSOR  THESE  CONFERENCES? 
BECAUSE  WE  BELIEVE  THAT  THE  COMMUNICATIONS  GENERATED  BY  THESE  GET-TOGETHERS 
IMPROVE  THE  ACCEPTANCE  OF  OUR  NEW  STANDARDS  AND  FOSTERS  EARLIER,  SUCCESSFUL 
IMPLEMENTATION  IN  NUMEROUS  APPLICATIONS.  WE  WANT  ALL  PARTIES  AFFECTED  BY 
THESE  STANDARDS  TO  KNOW  JUST  WHAT  IS  AVAILABLE  TO  SUPPORT  THEM:  THE 
HARDWARE;  THE  COMPLIANCE  TESTING;  THE  TOOLS  NECESSARY  TO  FACILITATE  DESIGN, 
ETC.  WE  ALSO  BELIEVE  THAT  FEEDBACK  FROM  PEOPLE  WHO  HAVE  USED  THEM  IS 
ESSENTIAL  TO  OUR  CONTINUED  EFFORTS  TO  IMPROVE  OUR  STANDARDIZATION  PROCESS. 
WE  HOPE  TO  LEARN  FROM  OUR  SUCCESSES  AND  OUR  FAILURES;  BUT  FIRST,  WE  MUST 
KNOW  WHAT  THESE  ARE  AND  WE  COUNT  ON  YOU  TO  TELL  US. 


AS  WE  DID  IN  1980,  WE  ARE  FOCUSING  OUR  PRESENTATIONS  ON  GOVERNMENT 
AND  INDUSTRY  EXECUTIVES,  MANAGERS,  AND  ENGINEERS  AND  OUR  GOAL  IS  TO 
EDUCATE  RATHER  THAN  PRESENT  DETAILED  TECHNICAL  MATERIAL.  WE  ARE  STRIVING 
TO  PRESENT,  IN  A  SINGLE  FORUM,  THE  TOTAL  AFSC  STANDARDIZATION  PICTURE  FROM 
POLICY  TO  IMPLEMENTATION.  WE  HOPE  THIS  INSIGHT  WILL  ENABLE  ALL  OF  YOU  TO 
BETTER  UNDERSTAND  THE  "WHY’S  AND  WHEREFORE'S"  OF  OUR  CURRENT  EMPHASIS  ON 
THIS  SUBJECT. 


MANY  THANKS  TO  A  DEDICATED  TEAM  FROM  THE  DIRECTORATE  OF  AVIONICS 
ENGINEERING  FOR  ORGANIZING  THIS  CONFERENCE;  FROM  THE  OUTSTANDING  TECHNICAL 
PROGRAM  TO  THE  UNGLAMOROUS  DETAILS  NEEDED  TO  MAKE  YOUR  VISIT  TO  DAYTON,  OHIO 
A  PLEASANT  ONE.  THANKS  ALSO  TO  ALL  THE  MODERATORS,  SPEAKERS  AND  EXHIBITORS 
WHO  RESPONDED  IN  SUCH  A  TIMELY  MANNER  TO  ALL  OF  OUR  PLEAS  FOR  ASSISTANCE. 


ROBERT  P.  LAVOIE,  COL,  USAF 
DIRECTOR  OF  AVIONICS  ENGINEERING 
DEPUTY  FOR  ENGINEERING 
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CV 

Second  PFSC  Standardization  Conference 


"  ASD/CC 

1.  Since  the  highly  successful  standardization  conference  hosted  by  ASD  in 
1980,  significant  technological  advancements  have  occurred.  Integration  of 
the  standards  into  weapon  systems  has  become  a  reality.  As  a  result,  we  have 
many  "lessons  learned"  and  oost/benef it  analyses  that  should  be  shared  within 
the  tri-service  community.  Also,  this  would  be  a  good  opportunity  to  update 
current  and  potential  "users."  Therefore,  I  endorse  the  organization  of  the 
Second  AFSC  Standardization  Conference. 

2.  This  conference  should  cover  the  current  accepted  standards,  results  of 
recent  congressional  actions,  and  standards  planned  for  the  future.  We  should 
provide  the  latest  information  on  policy,  system  applications,  and  lessons 
learned.  The  agenda  should  accommodate  both  government  and  industry  inputs 
that  criticize  as  well  as  support  our  efforts.  Experts  from  the  tri-service 
arena  should  be  invited  to  present  papers  on  the  various  topics.  Our  PFSC 
project  officer,  Maj  David  Hammond,  HQ  PPSC/KLR,  AOTOVON  858-5731,  is  prepared 
to  assist. 


R03ERT  M.  BOND,  Lf  Gen,  USA£ 
Vico  Commander 
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NAVY  CASE  STUDY 

IMPLEMENTATION  OF  MILITARY  STANDARDS 


Instructor:  Marshall  R.  Potter 

Naval  Electronics  System  Command 


ABSTRACT 


A  brief  overview  of  the  Navy  approach  to  the  life  cycle  management  of 
embedded  computer  resources  will  be  provided.  The  role  of  software 
engineering  as  a  problem  solving  discipline  involving  engineering,  computer 
science  and  management  will  be  applied  to  all  phases  of  the  life  cycle  as 
defined  in  DODD  5000.1.  A  case  study  of  a  major  Navy  acquisition  initiated  in 
1974,  subsequently  deployed  and  currently  under  maintenance,  will  be  covered 
and  analyzed.  The  purpose  of  the  case  study  is  to  investigate  a  system 
acquisition  that  utilized  the  most  up-to-date  technology  practical,  including 
recommended  software  development  tools,  techniques,  and  methodologies.  The 
usefulness  and  shortfall  of  good  tools  and  techniques,  employed  during  the 
acquisition  of  a  complex  system,  will  be  discussed  illustrating  that  things 
dont  always  turn  out  right,  even  when  prosecuted  in  accordance  with  the  best 
expertise  and  guidance  available. 


BIOGRAPHY 


Marshall  Potter 

Head,  Software  Engineering  Branch 
Naval  Electronics  System  Command 
Washington  D.  C.  20360 

BSEE  University  of  Maryland  1971 
MSEE  University  of  Maryland  1974 
MSCS  University  of  Maryland  1979 

Experience 


15  years  with  the  Department  of  Defense,  including  the  following  assignments: 
Naval  Ship  Research  and  Development  Center 
Defense  Communications  Engineering  Center 
Naval  Electronics  System  Command 

Current  Assignment:  Head,  Software  Engineering  Branch,  Computer  Resources 
Division,  Naval  Electronics  System  Command 

Responsible  for  developing  procedures  and  Dolicy  for  the  design  and 
implementation  of  systems  that  use  embedded  computer  resources. 
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IT  IS  NOT  SUFFICIENT  TO  SPECIFY  GOOD  SOFTWARE  DEVELOP¬ 
MENT  CRITERIA 


SPECIFICATION  SHEET 


NAVAL  ELECTRONICS  SYSTEM  CASE  STUDY 


TOPIC: 

Management  problems  and  solutions  associated  with  a  Naval  electronics 
embedded  computer  software  system  throughout  its  development  history. 

TYPE:  Case  Discussion,  1  1/2  Hrs. 

OBJECTIVES: 

.  To  provide  the  participant  with  an  exercise  in  analyzing  embedded 
computer  software  management  activities. 

.  To  provide  the  participant  with  an  opportunity  to  apply  his  own  skills 
and  experience  to  a  set  of  typical  problems  associated  with  virtually  every 
complex  software  intensive  system. 

.  To  provide  the  opportunity  for  participants  to  trace,  in  a  single 
thread  fashion  through  the  entire  development  cycle  of  a  system  as  it 
experiences  early  difficulty,  periods  of  constant  reassessment,  and  finally, 
success . 
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NAVAL  ELECTRONICS  SYSTEM 


DISCUSSION: 

In  the  fall  of  1974,  the  Navy  Project  Management  Office  -  Electronics 
(PME)  solicited  an  RFP  for  proposed  system  designs  for  a  naval  electronics 
system.  Of  the  original  38  contractors  who  attended  the  initial  briefing, 
four  teams  of  contractors  submitted  proposals.  Two  of  these  contractor  teams 
were  eliminated,  primarily  due  to  the  fact  that  their  proposals  disclosed  that 
the  contractors  did  not  understand  the  depth  and  complexity  of  the  require¬ 
ments.  The  remaining  two  contractors  were  selected  to  submit  system  specifica¬ 
tions  by  April  of  1975.  At  that  time,  two  contenders  were  placed  under  cadre 
tasking  while  their  proposals  were  evaluated.  Under  the  cadre  tasking,  the 
two  contractors  were  directed  to  further  refine  their  specifications  and  to 
develop  preliminary  Program  Performance  Specifications  (PPSs)  .  In  August  of 

1975,  the  PME  selected  one  company  as  the  prime  contractor  with  a  separate 
company  as  the  major  software  developer.  The  software  subcontractor  was 
directed  to  continue  development  of  the  PPS  and  the  Program  Design  Specifica¬ 
tion  (PDS).  The  contract  for  full  3cale  development  was  signed  in  March  of 

1976. 

Under  the  terms  of  the  contract,  the  software  subcontractor  was  required 
to  utilize  a  top-down  modular  design  methodology,  the  CMS-2  high  order 
language,  and  structured  programming  techniques.  As  top-down  implementation 
of  the  design  proceeded,  the  software  subcontractor  would  deliver  software  in 
four  basic  increments  resulting  in  a  final  delivery  of  an  integrated,  tested 
software  system  by  May  1977.  System  integration  of  software  to  hardware  was 
specified  for  completion,  with  acceptance  testing,  by  September  1977.  This 
allowed  a  period  of  only  18  months  from  the  start  of  full  scale  development  to 
completion  of  acceptance  testing  for  a  system  composed  of  18  hardware  racks 
and  associated  system  software;  a  very  ambitious  contract. 

The  PME,  realizing  that  they  had  a  lack  of  computer  software  trained 
personnel,  negotiated  with  the  Naval  Ocean  Systems  Center  (NOSC)  to  provide 
software  support  during  the  development  phase  of  the  program.  NOSC  is  a  major 
Navy  laboratory  that  i3  electronics-oriented.  They  experiment  in  microelec¬ 
tronics,  radar,  and  satellite  systems.  On  this  program,  NOSC  was  specifically 
tasked  to: 

1.  Provide  support  to  the  PME  during  the  review  and  evaluation  of 
contractor  produced  software  and  documentation . 

2.  Serve  as  the  single  point  of  contact  in  the  provision  of  software 
support  to  the  prime  contractor. 

3.  Provide  facilities  and  instruction  to  the  computer  system  hardware 
and  software  maintenance  agent,  the  Naval  Electronic  Systems  Engineering 
Center  (NESEC) . 

4.  Install  Level  2  Support  Software  (CMS-2M,  SDEX120)  at  the 
contractor  development  and  test  facilities. 

5.  Perform  testing  and  verification  of  incremental  software 
deliveries  and  report  results  to  the  PME. 
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In  support  of  these  requirements,  NOSC  would  partake  in  all  design  and 
program  reviews,  computer  program  regeneration ,  and  computer  program 
acceptance  demonstrations, 

NESEC,  San  Diego  was  designated  as  the  software  maintenance  agent.  It  was 
recognized  that  NESEC  would  have  to  participate  in  all  phases  of  development 
to  gain  the  necessary  experience  to  undertake  software  support  functions 
upon  completion  of  system  development.  NESEC  personnel  were  to  interface, 
primarily  witn  NOSC,  to  gain  the  needed  hands-on  experience. 

During  the  eleven-month  cadre  tasking  period,  the  software  subcontractor 
continued  to  refine  the  system  specifications  and  develop  the  PPS  and  the  PDS. 
A  preliminary  PPS  was  delivered  just  prior  to  the  signing  of  the  full  scale 
development  contract.  Within  90  days  of  the  contract  signing,  a  preliminary 
PDS  was  delivered.  No  realistic  new  cost  or  schedule  reestimation  was 
attempted . 

Upon  delivery,  the  PPS  and  the  PDS  were  submitted  to  exhaustive  design 
reviews.  The  PPS  document  was  determined,  with  minor  exceptions,  to  accu¬ 
rately  specify  system  requirements  but  was  not  in  the  format  required  by 
SECNAV  Instruction  3560.1.  Rewriting  of  the  PPS.  in  accordance  with  the 
instruction,  resulted  in  a  six-month  delay  in  delivery  and  acceptance  of  the 
final  document  with  corresponding  delay  in  placing  the  allocated  oaseline 
under  configuration  management  control.  The  initial  version  of  the  PDS  con¬ 
stituted  well  over  1000  pages  and  was  subsequently  determined  by  the  software 
contractor  to  specify  design  requirements  at  too  low  a  level  and  in  too  great 
detail.  The  final  document  was  on  the  order  of  200-300  pages.  A  significant 
shortcoming  of  the  PDS  was  that  it  did  not  provide  detailed  interface 
specifications  between  software  and  hardware. 

The  programming  phase  of  software  development  commenced  upon  initiation 
of  the  development  contract.  The  software  subcontractor  began  coding  before 
either  the  PPS  or  the  PDS  were  finalized.  In  July  of  1976,  it  became  apparent 
that  the  software  subcontractor  was  not  spending  funds  in  accordance  with  the 
Cost  and  Schedule  Control  Program  (as  required  by  DODD  7000.2).  This  initia¬ 
ted  an  investigation  that  indicated  that  the  software  developer  was  not 
meeting  the  scheduled  requirements  for  delivered  lines  of  executable  code. 
Although  reluctant  to  admit  to  development  delays,  the  software  subcontractor 
eventually  had  to  acknowledge  that  they  were  experiencing  significant  software 
production  difficulties  when  they  missed  delivery  of  the  second  software 
increment  in  November  of  1976.  In  December  of  1976,  a  number  of  actions  were 
recommended  to  contain  cost  growth  and  to  schedule  a  slip  in  the  program. 

Among  the  actions  taken  were  replacement  of  an  IBM  370/135  support  computer 
with  an  IBM  370/145,  partial  delivery  of  the  second  software  increment,  and 
significant  reorganization  of  the  software  subcontractor  management.  At  this 
time,  software  development  was  from  three  to  four  months  behind  schedule. 

With  the  partial  delivery  of  the  second  increment  of  software  in  February 
of  1977,  integration  of  software  to  hardware  began.  It  soon  became  apparent 
that  significant  problems  existed  in  accomplishing  the  integration  because  of 
inadequate  implementation  of  interface  controls  within  the  software.  As  inte¬ 
gration  difficulties  expanded,  more  and  more  resources  were  diverted  to  the 
test  site  to  contain  the  problem.  In  order  to  allow  more  time  to  deal  with 
integration  problems,  the  hardware  development  began  to  drive  the  software. 
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Even  though  software  development  was  three  to  four  months  behind,  tne  Navy 
refused  to  relax  the  schedule.  This  resulted  in  abandoning  incremental 
deliveries  in  favor  of  drop  deliveries  of  software  modules  which  would  inter¬ 
face  with  the  emerging  hardware  devices.  This  required  that  the  software 
subcontractor  reestablish  software  priorities  to  support  hardware  availabil¬ 
ity.  The  change  from  incremental  software  deliveries  to  drop  deliveries 
seriously  impacted  verification  and  the  test  center's  ability  to  provide  test 
results  in  a  timely  manner. 

At  the  same  time,  a  problem  of  simple  logistics  became  evident.  The  400 
mile  separation  between  the  test  site  and  the  software  development  personnel 
was  creating  additional  delays  in  development.  As  a  result,  in  December  1977, 
all  software  development  personnel  were  relocated  to  the  test  site  to  optimize 
the  integration  process. 

While  the  major  software  development  was  going  on  by  the  software  subcon¬ 
tractor,  NOSC  was  expending  considerable  resources  in  establishing  an  indepen¬ 
dent  verification  and  test  center  at  NESEC,  San  Diego.  A  test  facility, 
consisting  of  operator  consoles  and  AN/UYK-20  computers  with  supporting  peri¬ 
pherals,  was  set  up.  Test  drivers  which  simulated  the  various  system  hardware 
devices  were  written.  Software  increments  were  simultaneously  delivered  by 
the  software  subcontractor  to  NOSC  and  to  the  main  test  site  at  the  prime 
contractor's  facility.  Various  tests  conducted  on  software  delivered  by  the 
software  subcontractor  uncovered  errors,  especially  in  modules  which  inter¬ 
faced  to  hardware.  There  was,  however,  no  correlation  performed  with  errors 
discovered  at  the  main  test  site  so  it  could  not  be  determined  which 
proportion  of  errors  discovered  at  NOSC  accurately  reflected  errors  which  were 
caused  by  improperly  coded  test  modules.  Although  NOSC’s  test  facility  was 
similar  to  the  system  configuration  in  computer  and  peripherals,  the  front-end 
was  software  simulated.  Errors  discovered  in  the  system  were  often  duplicates 
of  errors  uncovered  by  the  prime  contractor.  Front-end  errors  were  often 
attributed  to  the  software  simulation  at  the  San  Diego  test  site. 

Although  the  San  Diego  test  center  did  perform  verification  functions,  tne 
primary  purpose  of  the  center  was  to  provide  training  to  NESEC  personnel .  Due 
to  the  high  priority  and  expedited  schedule  of  the  program,  there  was  not  the 
usual  three  to  four  year  test  phase  to  gain  knowledge.  Therefore,  the  test 
activities  at  NOSC  provided  invaluable  experience  for  the  software  maintenance 
personnel  from  NESEC. 

At  the  present  time,  system  deployment  is  on  schedule.  The  first  system 
was  fielded  three  years  following  the  start  of  full  scale  development.  The 
software  subcontractor  is  under  a  maintenance  contract  to  help  clear  "bugs" 
and  provide  training  to  NESEC  personnel.  Although  no  changes  were  allowed 
during  the  initial  production  due  to  the  tight  schedule,  approximately  100 
class  II  changes  were  presently  in  work  in  1978.  Most  of  these  changes  are  to 
improve  efficiency  by  revisions  to  the  display  format,  timing,  etc..  In  the 
summer  of  1979,  the  Navy  assimed  configuration  control  at  the  code  level.  All 
libraries  were  removed  from  the  prime  contractor  and  assumed  by  the  Navy. 

CONCLUSIONS; 

Given  the  scarcity  of  formal  guidance  and  the  lack  of  computer  software 
trained  personnel,  the  PME  did  a  commendable  job  in  advance  planning  and 
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utilization  of  computer  resources  for  the  project.  State-of-tne-art  design 
and  development  techniques  were  demanded  of  the  contractor  such  as  new  hard¬ 
ware,  new  support  software,  top  down  structure,  and  tne  latest  USN  standards 
and  specifications.  The  need  for  a  maintenance  support  agency  for  post¬ 
development  pnases  of  tne  software  life  cycle  was  recognized  early.  Foresight 
was  evident  in  insuring  that  the  maintenance  agency  would  receive  adequate 
training  ana  participate  in  all  aspects  of  development.  Acknowledging  their 
lack  of  software  expertise,  tne  PME  employed  NOSC  to  assist  in  providing  for 
quality  assurance  of  the  delivered  product.  The  use  of  a  high  order  language, 
structured  programming,  and  good  program  doc Lmentation  w3s  specified  to 
improve  ease  of  software  maintenance.  Personnel  from  the  user  community  were 
trained  and  incorporated  into  the  test  program.  Problems  in  production  were 
detecte.1  early  and  aggressively  attacked  by  the  project  management  group. 
Unfortunately,  advance  planning  and  close  program  monitoring  by  the  PME  were 
not  sufficient  to  prevent  a  significant  software  cost  overrun  and  schedule 
si i ppage  . 


The  two  predominant  causes  of  cost  overrun  and  schedule  slippage  were 
underestimation  of  the  task  complexity  by  the  major  software  contractor  and 
inadequate  planning  for  the  system  integration  by  the  prime  contractor. 
Discussions  with  the  software  subcontractor  disclosed  that  they  themselves 
felt  that  the  complexity  of  software  requirements  was  grossly  underestimated 
due  to  the  utilization  of  a  relatively  naive  engineering  team  to  initially 
scope  tne  problem  anc  design  the  system.  Specifications  resulting  from  this 
effort  were  not  adequately  reviewed  by  higher  level  management  within  the 
software  subcontractor  organization  and  were  not  balanced  against  preliminary 
cost  and  time  estimations.  Software  subcontractor  management  admitted  that 
this  could  have  been  accomplished  by  project  management  personnel  if  adequate 
funds  and  time  had  been  provided.  The  assumption  that  the  underestimation  of 
program  complexity  was  a  main  problem  is  borne  out  by  growth  in  source  code 
and  by  code  production  figures  for  the  project  which  show  that  lines  of  code 
produced  per  manhour  were  slightly  above  average  for  the  industry.  It  can 
also  be  shown  that  the  software  subcontractor' s  original  budget  for  cost  per 
executable  Lines  of  code  called  for  a  production  rate  well  above  the  norm.  In 
spite  of  tr.ese  problems,  tne  delivered  code  is  generally  of  good  quality  with 
tne  exception  of  the  interface  software  controls. 

The  software  and  Hardware  developing  agencies  must  both  be  held  account¬ 
able  for  integration  of  software  to  hardware.  If  the  interface  is  designed 
properly,  there  is  no  good  reason  why  hardware  and  software  developers  cannot 
meet  specifications  which  will  promote  a  smooth  integration.  Much  of  the 
delay  that  was  realized  in  s;  ..tem  integration  for  the  project  was  due  to  the 
failure  to  produce  rigid  interface  design  specifications  early  in  the  project 
and  plan  them  under  configuration  control.  Another  cause  of  delay  in  system 
integration  was  the  lack  of  planning  for  this  phase  of  development.  The  prime 
contractor,  who  was  tasked  with  final  responsibl ity  for  system  integration, 
failed  to  provide  a  functional  integration  pi  ar. .  They  allowed  only  33 
manmonths  for  the  software  developer  to  accomplish  its  portion  of  the  task. 

Tne  emergence  of  various  items  of  hardware  to  meet  Navy  schedules  even  though 
software  was  late,  was  permitted  to  become  the  main  driving  force  behind  the 
path  of  integration  thereby  pertubating  the  software  development  plan  and 
further  aggravating  the  3chf.dule  for  delivery  of  software. 
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•Jtner  fac  i  .•  r  s  wr; .  -t.  impuo  ted  s- ft ware  cost  ana  delivery  includes:  failure 
to  finalize  cue  ;t  ^  a:u  the  PD3  oefc-re  onmencement  of  coding;  lack  of 
adequate  test  ;ua:i:  ,  spec i ideations ,  anc*  Procedures,  and  lack  of  well-defined 
acceptance  criteria;  lack  of  AN/IJYK-20  software  support  diagnostic  and 
debugging  aide:  and  .sok  of  software  engineering  personnel  within  the  PME. 
Failure  to  f .  .  i . .  ze  tne  rPS  and  tne  PDS  before  the  start  of  programing 
permitted  tne  incorporation  of  design  errors  into  the  program  which  proved 
very  costly  to  e.wv.nate  at  a  later  date.  In  addition,  problems  were  often 
pushed  off  fur  liter  resolution  ana  often  resulted  in  unforeseen  ramifications 
in  otner  program  .-nucules.  For  tne  same  reasons,  inadequate  test  plans  and 
procedures  can  prove  ustly  if  tne  program  is  not  tested  properly  to  insure 
that  all  requirements  are  being  met.  An  early  definition  of  good  test  specifi¬ 
cations  and  acceptance  criteria  will  assist  the  developer  in  under  stand ing  the 
requirements  and  motivate  nim  to  deliver  a  product  which  meets  acceptance 
goals.  Tne  inadequate  test  plans  on  this  project  substantiate  the  fact. 
Frequently,  the  software  subcontractor  nad  no  concept  of  an  adequate  accept¬ 
ance  criteria  for  their  software  modules.  The  quality  of  documentation 
between  the  prime  contractor  and  the  subcontractor  was  poor.  Although  the 
software  subcontractor  requested  more  definitive  procedures,  the  prime 
contractor  did  not.  Since  this  program  was  one  of  the  first  to  use  the 
AN/UYrC-dO  computer,  tne  Navy  lacked  support  software  to  support  structured 
code.  This  forced  the  contractor  to  direct  resources  to  develop  the  necessary 
tools.  Although  the  Navy  was  driving  the  use  of  structured  codes,  the  Navy 
was  not  ready  to  support  these  types  of  systems . 


Another  issue  implicit  in  the  study  of  this  project  is  the  lack  of  land 
based  test  sites  within  this  Navy  organization.  Such  sites  can  provide 
facilities  for  software  maintenance,  independent  acceptance  testing,  ar.d 
training  for  both  support  agency  personnel  and  the  user  community.  The  PME 
had  to  develop  its  own  maintenance  and  test  site  to  meet  the  project 
requirements . 

The  desirability  of  a  verification  and  test  center,  such  as  furnished  oy 
NOS C,  in  major  software  development  programs  is  questionable.  The  requirement 
to  verify  all  computer  programs  as  they  are  delivered  levies  a  significant 
software  development  task  on  the  test  center  that  must  parallel  the  efforts  of 
the  major  developing  agency.  It  is  not  clear  whetner  the  test  center  might 
not  either  substantially  delay  the  whole  program  or  be  entirely  bypassed  if 
they  do  not  produce  on  schedule.  Moreover,  if  errors  were  discovered,  it 
would  not  be  readily  apparent  that  they  had  not  been  introduced  by  the  test 
center  through  test  drivers  and  simulators.  What  is  clearly  needed  is  an 
agency  to  provide  support  in  the  areas  of  software  engineering,  program 
moni tor ing ,  and  test  plan  and  acceptance  criteria  development.  It  would  be 
highly  desirable  if  this  support  could  be  provided  by  the  designated  post¬ 
development  software  maintenance  agency  who  has  a  vested  interest  in  the  final 
product.  Any  proposed  agency  of  this  type  should  be  under  the  control  of  the 
PME.  Both  the  PME  and  any  post-development  software  maintenance  agency  would 
be  highly  motivated  to  insure  adequate  system  configuration  and  implementation 
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QUESTIONS 

1.  Based  on  the  history  of  this  project,  discuss  the  pros  and  cons  of  a 
separate  test  agency. 

2.  Discuss  the  various  pitfalls  of  a  project  such  as  this  one  where 
development  and  production  run  simultaneously  in  order  to  meet  critical 
sched  ules . 

3.  What  steps  should  a  project  management  office  take  when  a  contractor  is 
obviously  producing  poor  or  inadequate  software  estimates? 

4.  Discuss  the  problems  that  occurred  and  the  result  of  the  Navy  requesting 
structured  codes  before  they  had  the  capability  to  support  such  systems. 

5.  Discuss  the  problems  that  can  occur  when  hardware  and  software  are 
developed  concurrently. 

6.  Discuss  the  steps  vrfiich  you  would  take,  given  the  mission  and  role  of  the 
PME,  to  preclude  the  management  problems  indicated  in  the  case  study. 
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